Power Analysis
Jasper Slingsby
Power Analysis
No one ever does them…
…but they could save so much pain and suffering if they did!!!
Power Analysis
Statistical power is the probability of a hypothesis test finding an effect if there is an effect to be found.
Power analysis is a calculation typically used to estimate the smallest sample size needed for an experiment, given a required significance level, statistical power, and effect size.
- It is normally conducted before the data collection!
Why do power analysis?
Firstly, it helps you plan your analyses before you’ve done your data collection, which is always useful.
Secondly, not knowing the statistical power of your analysis can result in
- missed findings (through Type II Error), or
- false findings (through Type I Error).
Why do power analysis?
Type II Error:
- occurs when the researcher erroneously concludes that there is not a difference between treatments, when in reality there is…
- this is a common outcome of low statistical power
Why do power analysis?
Type I Error:
- occurs when the researcher erroneously concludes that there is a difference between treatments, when in reality there is not…
- less likely when there is poor statistical power, but can happen with low sample sizes of highly variable subjects, or if there is bias in sampling…
Why do power analysis?
![]()
Type I and Type II Errors and how they result in false or missing findings, respectively. Image from Norton and Strube 2001, JOSPT.
Statistical Power
Is determined by the combination of the:
- \(\alpha\) (“significance”) level required (e.g. P < 0.05)
- difference between group means (effect size)
- variability among subjects
- sample size (the factor we usually have most control over)
\(\alpha\) (“significance”) level
![]()
We usually use an \(\alpha\) of 0.05 to indicate significant difference.
- i.e. the probability of the observation not being different to the null is less than 5% (i.e. p < 0.05), or the result should only be observed once or less for every 20 samples.
This is a subjective cut-off, but is generally accepted in the literature…
Difference between group means
![]()
You have greater statistical power when you have greater differences in means (effect size). P1 vs P3 has greater power than either P1 vs P2 or P2 vs P3.
Variability among subjects
![]()
Greater variability among subjects results in larger standard deviations, reducing our ability to distinguish among groups (i.e. statistical power).
Sample size
![]()
Increasing sample size increases statistical power by improving the estimate of the mean and constricting the distribution of the test statistic (i.e. reducing the standard error).
…
![]()
Bullets
When you click the Render button a document will be generated that includes:
- Content authored with markdown
- Output from executable code
Code
When you click the Render button a presentation will be generated that includes both content and the output of embedded code. You can embed code like this:
